A related application entitled Layered Representation For Image Coding, Ser. No. 07/888,801 to Edward H. Adelson, which is incorporated herein by reference, describes an image compression system which represents an image as a series of layers ordered by "depth" in the image. This system represents an image sequence by including in each of the layers information relating to how that layer should be manipulated, or transformed, over the time span of the image sequence.
A layer consists of a series of data maps, each of which relates to an object or a portion of an object which has motion which is significantly different from that of any other object in the image. Each map consists of a set of data for discrete two dimensional locations, with some maps including time as a third dimension. The maps define for each location (i) the intensity at a fixed instant in time, for example, the start of a sequence, (ii) the attenuation over time, and (iii) the velocity over time. Optional maps, such as a contrast change map which describes how the intensity map of the corresponding layer should be multiplied to create contrast transitions over time, a blur map which adds motion blur or focus blur to one or more locations, and so forth, may be included, as necessary, to describe changes in the associated objects over time. The image sequence is re-created, or decoded, from the layers by combining them, in order, and appropriately manipulating, i.e., "warping," them over time.
Various methods may be used to define the objects, i.e., to determine their boundaries. One may use recursive multiple motion analysis, one example which is a dominant motion analysis technique, which iteratively assumes there is a single dominant motion in an image sequence and based on that assumed motion defines a single (dominant) object. This dominant object is then masked or excluded from the image, and the remaining regions of the image are again analyzed, assuming a second dominant motion. If a second object is identified, it too is excluded from the image and the image can again be warped and subjected to further motion analysis. This type of dominant motion analysis is discussed in a paper entitled Image Sequence Enhancement Using Multiple Motions Analysis, by Irani and Peleg. The technique correctly identifies a dominant object if there is a single, clearly dominant motion in the image. If, however, there are multiple dominant motions in the image, the technique cannot accurately identify the objects.
Another technique for determining objects in an image is block matching. The motions of rectangular blocks of pixels are determined from one frame to another and motion vectors are assigned to the blocks. Blocks with the similar motions are then considered parts of the same object. All objects, even objects with irregular shapes, are represented by a combination of these blocks. While blocks of varying sizes may be used to designate an object, as discussed in a paper entitled Segmentation--Based Coding of Motion Difference and Motion Field Images For Low Bit-Rate Video Compression, by Liu and Hayes, these blocks cannot entirely accurately represent the irregularly shaped objects.